Image capture and display device

ABSTRACT

A display device for capturing and displaying images along a single optical axis, having an image capture device for capturing the objective image through the display panel when the display device is in a second transmissive state; an image supply source for providing an image to a display panel when the display panel is in a first display state; a mechanism for alternating placing the display panel between the first display state and second transmissive state such that an image can be viewed on the display screen and the object can be captured such that the alternating between the first display state and the second transmissive state is substantially imperceptible to a user of the display panel; and a mechanism for providing digitally image processing for captured images prior to display.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a divisional of U.S. patent application Ser. No. 12/178,854filed Jul. 24, 2008, which is a continuation of U.S. patent applicationSer. No. 10/911,281 filed Aug. 4, 2004, now U.S. Pat. No. 7,420,585,which is a continuation-in-part of U.S. patent application Ser. No.10/264,058 filed Oct. 3, 2002, now U.S. Pat. No. 7,042,486, which is acontinuation of U.S. patent application Ser. No. 09/452,091 filed Nov.30, 1999 (abandoned).

Reference is made to commonly-assigned, co-pending U.S. patentapplication Ser. No. ______ (Kodak Docket 77734DUS05) filed concurrentlyherewith, entitled “Image Capture and Display Device”, by John R.Fredlund, et al, the disclosure of which is incorporated herein.

FIELD OF THE INVENTION

The present invention relates to an electronic image capture and displaydevice, and in particular an image capture and display device of thetype employing an electronic camera to capture an image of a subject andan electronic image display device to display images.

BACKGROUND OF THE INVENTION

U.S. Pat. No. 4,864,410, issued Sep. 5, 1989 to Andrews et al.,illustrates a technique using a beam splitter, electronic camera, and avideo display (CRT) to simultaneously capture and view a subject on thesame optical axis similar to the function of a conventional opticalmirror. The images captured by the camera can be processed prior todisplay to provide special effects such as the distortions employed in afun-house mirror. The problem with this technique is that the displayedimage appears to be located at a distance behind the beam splitter,compromising the “mirror-like” function of the unit. In addition, thevideo display device must be masked off or it may be viewable by theuser also detracting from the “mirror experience”. It is well known anddesirable to use a touch screen as a means for providing an interfacebetween a user and a display device such as a CRT, however with thearrangement described by Andrews et al., the interface with a touchscreen located between the user and the beam splitter does not provide asatisfactory feel, since when the subject places her finger on the touchscreen, the image of the subject's finger does not appear to be locatedat the screen, rather it appears to be displaced to the rear of thescreen.

Using a conventional rear projection screen and a digital projectorwould provide a display image that would appear in a viewing plane likea conventional mirror, and would function in a satisfactory manner witha touch screen interface, but a conventional rear projection screenwould prevent the camera from “seeing” the subject, unless a hole wasprovided in the center of the screen. This is not a very satisfactorysolution.

U.S. Pat. No. 5,801,758 issued Sep. 1, 1998 to Heirich discloses asystem for image capture and display. The system includes an imagecapture apparatus, an image projection apparatus, and an opticalswitching device that switches an optical path between the image captureand the image projection apparatus. A highly transparent surface, suchas a sheet of clear glass is used as a display screen through which theimage capture apparatus can view a user, and onto which the image of theuser is projected. The problem with this arrangement is that the highlytransparent surface allows the user to directly view the projection lensof the image projection apparatus. Any image formed on the highlytransparent surface is overwhelmed by the bright appearance of theprojection lens.

Another problem in the prior art is the ability to capture an image ofperson on the same optical axis on which the person is viewing a displaydevice. For example, with regard to video conferencing, a capture camerais positioned at an optical axis different from which the person isviewing the other individual on the video conference call. The problemwith the described situation is that the person viewing the displaywhile being recorded by an off-axis camera appears to fail to make eyecontact with the recording camera and whoever may be viewing that image.There is a similar problem with an individual using a teleprompter. Aperson reading from a teleprompter is looking in a direction off axisfrom the optical axis of a camera capturing the image of the reader.

There is a need therefore for an improved image capture and displaydevice that avoids the shortcomings of the prior art.

SUMMARY OF THE INVENTION

In accordance with one aspect of the present invention there is provideda display device for capturing and displaying images along a singleoptical axis, comprising:

a display panel having a front side and a back side, the display panelcapable of being placed in a first display state which allows displayingof an image on the front side for viewing of an image and a secondtransmissive state wherein the display panel becomes substantiallytransmissive so as to allow capture of an image of an object positionedin front of the front side of the display panel;

an image capture device for capturing the objective image through thedisplay panel when the display device is in the second transmissivestate;

an image supply source for providing an image to the display panel whenthe display panel is in the first display state; a mechanism foralternating placing the display panel between the first display stateand second transmissive state such that an image can be viewed on thedisplay screen and the object can be captured in a manner such that thealternating between the first display state and the second transmissivestate is substantially imperceptible to a user of the display panel; and

a mechanism for providing digitally image processing for captured imagesprior to display.

In accordance with another aspect of the present invention, there isprovided a display device for capturing and displaying images along asingle optical axis, comprising:

a display panel, includes means for causing becomes substantiallytransmissive so as to allow capture of an image of an object positionedin front of the front side of the display panel;

an image capture device for capturing the objective image through thesubstantially transmissive display panel; and

an electronic, optical, or computational means for compensating for anynon-transmissive elements of the display panel so that the presentedimage appears without artifacts caused by non-transmissive elements.

In accordance with yet another aspect of the present invention, there isprovided a display device for capturing and displaying images along asingle optical axis, comprising:

a display panel having a front side and a back side, the display panelcapable of being placed in a first display state which allows displayingof an image on the front side for viewing of an image and a secondtransmissive state wherein the display panel becomes substantiallytransmissive so as to allow capture of an image of an object positionedin front of the front side of the display panel;

an image capture device for capturing the objective image through thedisplay panel when the display device is in the second transmissivestate;

an image supply source for providing an image to the display panel whenthe display panel is in the first display state;

a mechanism for alternating placing the display panel between the firstdisplay state and second transmissive state such that an image can beviewed on the display screen and the object can be captured in a mannersuch that the alternating between the first display state and the secondtransmissive state is substantially imperceptible to a user of thedisplay panel; and

a touch screen positioned over the display panel.

These and other aspects, objects, features and advantages of the presentinvention will be more clearly understood and appreciated from a reviewof the following detailed description of the preferred embodiments andappended claims and by reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In the detailed description of the preferred embodiments of theinvention presented below, reference is made to the accompanyingdrawings in which:

FIG. 1 is a schematic diagram of an image capture and display deviceaccording to the present invention;

FIG. 2 is a schematic diagram of a modified image capture and displaydevice made in accordance with the present invention;

FIG. 3 is a schematic diagram of another modified image capture anddisplay device made in accordance with the present invention; and

FIG. 4 is a schematic diagram of yet another modified image capture anddisplay device made in accordance with the present invention

FIG. 5 is a schematic diagram of still another modified image captureand display device made in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIG. 1, the image capture and display device, generallydesignated 10, according to the present invention is housed for examplein a cabinet 12. It will be understood that the capture and displaydevice may be a component of a larger device such as a photobooth (notshown). An electronic camera 14 such as a KODAK DIGITAL SCIENCE DVC323Digital Video Camera available from the Eastman Kodak Company,Rochester, N.Y., and a display panel 16 such as an OLED panel availablefrom the Eastman Kodak Company, Rochester, N.Y., are located in thecabinet 12 with respect to an optical element such as a fully silveredmirror 18 in a known manner such that they share a common optical axis20. The display panel 16 is positioned in a front opening 13 of thecabinet 12. Display panel 16 is electronically switchable between afirst state and a second state. Display panel 16 may comprise forexample a matrix of Organic Light Emitting Diodes that change state whenan electrical current is applied. In the “power off” state (transmissivestate), the material is substantially transparent and suitable forviewing there through. In the “power on” state, the display panel 16will display an image for viewing by an individual (viewer) 24positioned in front of display panel 16. The interior and contents ofthe cabinet 12 is unlighted and preferably painted a matte black so thatwhen the display panel 16 is in the “power off” state, an individual 24does not see the interior of the cabinet. In a preferred embodiment, thebrightness of display panel 16 is such that any small amount of lightwithin cabinet 12 will go unnoticed. It is preferred to use a camera ofhigh sensitivity so that the ambient illumination of the user isadequate for high quality image capture. An optional lamp may beprovided for illuminating the user 24 during image capture by camera 14.Lamp 30 is turned on only during image capture by camera 14 so as tominimize interference with the image on display panel 16, and tominimize the light impinging upon the eyes of user 24.

In the preferred embodiment, display panel 16 is fully transparent sothat it does not interfere with the capture of the image of user 24 bycamera 14. If this is not the case, and only a portion of display panel16 is transparent, the portions of display panel 16 that are nottransparent must be taken into account. For example, if the transistorsused to turn individual transparent OLED devices on and off arethemselves not transparent, the result is that only the portions ofdisplay panel 16 that comprise the OLED devices will allow capture ofuser 24 by camera 14. The effect will be that of looking through a finepitch screen. Thus, the screen must be interpolated out of the capturedimage, or the image must be somewhat blurred by electronic, optical, orcomputational means so that the presented image of the user appearswithout the screen artifact.

An optional touch screen 23 may be provided over display panel 16. Thetouch screen 23 can be of the type having an array of infraredtransmitters and detectors arranged around the periphery of the displaypanel 16, or a transparent pressure sensitive panel disposed over thedisplay panel 16. The display panel 16 is located with respect to commonoptical axis 20, for allowing the electronic camera 14 to capture theimage of a subject 24 through the display panel 16 when in thetransparent state and for displaying an image by the digital displaypanel 16 for viewing by individual (user) 24 along a viewing axis. Inthe embodiment illustrated, digital display panel 16 comprises OLEDdevice. An example of a suitable organic light emitting diode (OLED)display device/panel for practicing the present invention is illustratedin the Kodak EasyShare LS633 zoom digital camera.

Returning to FIG. 1, control of the image capture device 14 and displaydevice 16, and digital image processing is provided by a computer 26,such as a standard microprocessor or personal computer. The output ofthe camera 14 and the input to the digital display panel 16 areconnected and controlled by computer 26. The computer 26 can function asboth a controller and/or as an image processing computer. It will beunderstood that a separate controller and digital image processingelectronics can be provided. A digitally controlled driver 28 isprovided for driving the display panel 16 to switch it back and forthbetween its first transparent state and second viewing state.

In operation, the computer/controller 26 and driver 28 first switchesthe display panel 16 to its transparent state by signaling driver 28 tode-energize the screen, then captures an image from camera 14, and thenswitches the display panel 16 to its viewing state, and back again via asignal to driver 28 to the display device 16, making it transparent.This cycle is then repeated preferably at a rate of at least 24 completecycles per second to provide the appearance of a smooth and “flickerfree” display. That is, in a manner such that said alternating betweensaid first display state and said second transmissive state issubstantially imperceptible to a user of said display panel. Thecaptured images may be processed by the computer 26 prior to displayusing known image processing techniques. The capture image may beforwarded to second image capture and display device 10′ (not shown) forviewing by another party and capture of the other party for view by theindividual at the first device 10. The second device 10′ beingsubstantially identical in performance as the first device 10. In thisway there may be two way video conferencing wherein both parties arelooking at the display device thereby providing a more pleasingon-screen appearance for both parties. The display is more pleasingbecause both users 24 and 24′ (not shown) are looking directly at thedisplay of the other user and effectively making eye contact since thedisplay and capture device are on the same optical access.

Alternatively, the device 10 may be operated in a manner to provide amirrored image of the individual in front of display panel 16. Themirrored image may be modified to incorporate desired effects, forexample, the image may be geometrically distorted as in a fun housemirror. Alternatively, background scenes may be added using known bluescreen techniques, colors may be modified or pre-stored image elements,such as mustaches, hats, glasses etc. may be digitally added. Alsoarticles of clothing may be added to the image of the subject, forexample to show how the subject would appear in the article of clothing.Note that the processed image does not need to immediately follow thecapture, but may be delayed by several frames to allow time for thedigital processing to be preformed.

Referring to FIG. 2 there is illustrated a schematic view of a modifieddevice 100 wherein like numerals indicate like parts and operation aspreviously described with respect to device 10. In this embodiment,display panel 16 comprises a liquid crystal display (LCD) 116 isprovided in place of an OLED. In this embodiment, a diffuser 112 is usedfor providing a substantially uniform light source for illuminating LCD116. When LCD 116 and diffuser 112 are in the off non-energized state,they are at least partially transparent so the camera 14 can capture animage of the individual 24. This image will have the pattern associatedwith the color filter array of the LCD. This must be taken into accountwhen processing the captured image. When camera 14 is off and it desiredto display an image on LCD 116, the LCD 116 and diffuser 112 are in theenergized state. The LCD 116 has pixels that can be turned on so thatvarying levels of light will pass through the pixels to form an image.Preferably red, green, and blue pixels are provided so that color imagemay be generated on LCD 116. Light from lamp 30 is used to backlight theimage formed by LCD 116 and is diffused by diffuser 112. In operationthe LCD 116 diffuser 112 and lamp 30 are cycled between the energizedand de-energized states with the camera capturing an image during thede-energized state and an image being displayed during the energizedstate. The lamp 30 may also be used to illuminate the user for at leasta portion of the time the camera captures an image. Here as with theOLED, the rate of cycling between the energized and non-energized stateis at a rate so that “flicker” of the image is not perceived by theviewer. A rate of at least 24 or more cycles per second is preferred.

Preferably during capture, the pixels of each color area are placed inthe condition where they pass the maximum amount of light, but thismaximum will be a wavelength specific according to the color of thepixel. Thus, the image captured will have the effect of this arrayimposed upon the viewer 23. One solution to this problem is to providethe LCD array with high resolution and camera 14 with low resolution. Inthis fashion, the individual pixels of the LCD array 116 blur togetherto create an image that does not appear to be captured through the LCDarray. Alternatively a blurring filter in camera 14 may achieve the sameresult. Another solution is to take advantage of the known pattern ofthe LCD array 116. In this case, camera 14 can be color or monochrome,and of a resolution high enough to resolve the pixels of the LCD array116. The image of user 23 is overlaid with the color pattern of LCDarray 116. When the image of user 23 is captured with a color camera,the missing values of red, blue, and green must be interpolated. In thecase of monochrome capture, red, green and blue pixel values aredetermined from known positions of the colored pixels, and then themissing values can be calculated from the monochrome capture.Registration or calibration markings may be provided to easecomputational complexity. Other color combinations such as cyan,magenta, and yellow can be used.

Referring to FIG. 3, there is illustrated a schematic of yet anothermodified capture and display device 200 similar to devices 10 and 100,like numerals indicating like parts and operation as previouslydescribed. In this embodiment of device 200, there is provided arotating diffuser 212 that is provided in front of lamp 30 for not onlydiffusing the light but also acting as shutter for allowing light fromthe lamp 30 to be directed to the LCD 116. A motor 214 controls the rateof rotation of diffuser 212 and is synchronized with the turning on andoff of the LCD 116.

Referring to FIG. 4 there is illustrated a schematic of still anothermodified capture and display device 300 similar to devices 10, 100, and200 like numerals indicating like parts and operation as previouslydescribed. In this embodiment lamp 30 provides the light for backlighting LCD array 116. A collimator 312 is provided for collimating thelight before reaching a half silver mirror 314. The light reflected offmirror 314 is displayed on LCD diffuser 112 during the image displaycycle. During the image capture portion of the cycle, the diffuser 112is changed to its transparent state so as to allow capture of an imageby cameras 320 behind mirror 314. Collimator 312 is essential to thisembodiment so that the light passing through every pixel in LCD 116 istraveling in a parallel manner to light one small portion of thediffuser 112.

Referring to FIG. 5 there is illustrated a schematic of still anothermodified capture and display device 400 similar to devices 10, 100, 200and 300 like numerals indicating like parts and operation as previouslydescribed. In this embodiment, camera 14 is placed directly on theoptical axis of device 400 so that user 24 is looking straight into thecamera as he observes the display on diffuser 112. The sequence ofdisplay and capture is as described above. Projector 34 illuminatesdiffuser 112 during the display portion of the cycle. Projector 34 isplaced off the optical axis of device 400 so that user will not perceivea hot spot in the illumination of diffuser 112. Projector 34 is placedsuch that there is not a direct line from projector 34 through diffuser112 to user 24. In a preferred embodiment, the light from projector 34strikes diffuser 112 from above. It is unlikely that user 24 will viewdisplay device 400 from below the optical axis, and there will be noperception of a hot spot. Mirrors (not shown) may be used to reduce thedepth of device 400.

The invention has been described in detail with particular reference tocertain preferred embodiments thereof, but it will be understood thatvariations and modifications can be effected within the scope of theinvention.

PARTS LIST

-   10 image capture and display device-   10′ second image device-   12 cabinet-   13 front opening-   14 electronic camera-   16 display panel-   18 fully silvered mirror-   19 partially silvered mirror-   20 common optical axis-   21 light-   22 light valve projection screen-   23 touch screen-   23 viewer-   24 subject-   24′ user-   26 computer-   28 digitally controlled driver-   30 optional lamp-   30 mechanical shutter-   32 motor-   34 projector-   34 optical panel-   36 liquid crystal device-   38 transparent substrate-   40 transparent substrate-   42 liquid crystal material-   44 transparent conductive layer-   46 transparent conductive layer-   48 alignment layer-   50 alignment layer-   52 alignment arrow-   54 alignment arrow-   56 polarizer-   58 polarizer-   60 electrical lead-   62 electrical lead-   100 modified device-   110 LCD-   112 diffuser-   116 LCD-   200 display device-   212 rotating diffuser-   214 motor-   300 display device-   312 collimator-   314 silver mirror-   320 cameras-   400 display device

1. A display device for capturing and displaying images along a singleoptical axis, comprising: a display panel having a front side and a backside, the display panel capable of being placed in a first display statewhich allows displaying of an image on the front side for viewing of animage and a second transmissive state wherein the display panel becomessubstantially transmissive so as to allow capture of an image of anobject positioned in front of the front side of the display panel; animage capture device for capturing the objective image through thedisplay panel when the display device is in the second transmissivestate; an image supply source for providing an image to the displaypanel when the display panel is in the first display state; a mechanismfor alternating placing the display panel between the first displaystate and second transmissive state such that an image can be viewed onthe display screen and the object can be captured in a manner such thatthe alternating between the first display state and the secondtransmissive state is substantially imperceptible to a user of thedisplay panel; and a touch screen positioned over the display panel.